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ABSTRACT 

A study using the static-group research design was performed 
comparing the raw standardized Stanford 9 high stakes test scores of career 
and technical education (CTE) students with those of other students while 
controlling for extraneous variables such as learning styles, special 
populations, gender, race, and ethnicity. Two thousand high school students 
in three Arizona school districts (urban, suburban, and rural) were in the 
study. The Stanford 9 test instruments used were developed for the Arizona 
Department of Education and are given to all Arizona students. The "Learning 
Styles" assessment was given to the students participating in the study. Test 
scores on the Stanford 9 were analyzed using the Statistical Package for the 
Social Sciences (SPSS 10.0) using frequencies, means, standard deviations, 
correlations, and regression. Students receiving Individualized Vocational 
Education Plans had higher Stanford 9 tests than those eligible but not 
receiving them. Students with CTE concentrations were not significantly 
associated with higher or lower test scores. The researchers concluded that 
CTE students will always do worse on raw score comparisons, but when . 
extraneous variables are controlled for, there is no difference between CTE 
and other students. Consequently, they concluded, raw score comparisons are 
inappropriate because the groups are different. (Contains 14 references and 
an extensive glossary) (SLR) 
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A Comparison between Career and Technical Education and Other Students 

on a High Stakes Test 

Abstract 

^ «« comparison a fair and accurate meaaum between 
he two groups? me answer ts ■‘No, Career and Technical Education (CTE) students Lor^ 
lower than otter students on a recent high stakes test in Arizona.” Yet, when otter Mue„«s 
were conUolled for the following factors were found to have a signific;,t influence on I s^L- 
^1 five Special PopulaUon areas (handicapped, limited English proficiency economically 
disadvMttged, academically disadvantaged and being a single patent) were significantly ^ 
^sommedwttt lower test scores and were predominantly founJ in tte CTE poTuS'^ Higher 

■ iI^T" Auditory Learner (leam by heating) scores were significantly 

^ornmed with higher test scores and were predominantly found with otter studltt Sm 

^nesthenc U™er scores were significantly associated with lower test scores and were 
predoimi^tly found with CTE students. Black, Hispanic or otter males were assormed with 
lower test scores. Hispanic females were associated with lower test scores. 

Therefore, after controlling for the otter influences (extraneous variables), no difference was 
found between the two groups. That means that they ate just different groups of people and a 

raw score companson IS not an appropriate comparison. peopteanoa 

to their special population status affect the comparison’ Yes 
b^me there was a higher ptopomon of CTE students who received special popufation services 
and because special populaUon categories were associated with lower test scores. 

The conclusions are simple: Career and technical education students, for the most part will 

iSetiT' Tn appropriate extraneous variables are built 

into the equanon and controlled for, there usually is no difference between CTE and otter 

testscores. The raw score comparisons are inappropriate because the 
poups are different. The differences m scores can be attributed to the effects of the extraneous 
variables and not because of curriculum choice. 

The implications for educators are important. If extraneous effects are not understood and 
controlled for, than career and technical education will not have a veiy positive future. 

Therefore the recoi^endations are: CTE Administrators and Teachers must understand the 
problems associated with raw score comparisons on standardized tests. CTE state leaders must 
utilize this type of information in CTE promotional campaigns. 

Introduction 

“High stakes test-based reform is an approach that is most often driven by state-level mandate 
Md It suits the political appetite for rapid, quantifiable results (Thompson, 2001). In 1983 A ’ 
Nation at Risk was introduced as a reform effort for the public school system. It was soon’ 
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followed by yet another effort of reform in 1 987 with America 2000. However, this was not the 
end of the refom^sault on education. The Clinton Administration introduced Goals 2000, 

which marks the third major or national attempt at reform for the nation’s education system 
Within the last two decades. 



Assessment has become a part of life, whether it is in the classroom or in the job field We as a 
society haw evolved our system of life around successes and failures. The task of assessment is 
i^ed everyday by numerous people. The teacher uses assessment to determine what the students 
have learned or what they have not learned (Brown, 1996). The parent uses assessment in the 
fashion of examination of chores to determine if their children are to receive their allowance. The 
employer uses assessment to gauge the progress of an employee. The employer may also use 
assessment to determme promotions of employees (Race, 1995). 

It may be a task that we have, become accustomed to, but is it something that we are comfortable 
wth. Is It necessary. Who shall determine the assessment and who will achieve at a higher rate'? 
These are questions that are often thought about, but are left for pondering. 

The state of Anzona uses standardized tests to determine the progress of students at the 
^condary level. Standardized tests vary in their ability to assess the student in a fair manner. 

These types of tests are used to assess higher order reasoning skills and academic growth over 
time (Sanders & Horn, 1995). 

Comparison is the main issue with standardized testing. It gives some way of determining the 
learning ability or retention of the student. These tests are often considered the most visible 
indicator of education successes and failures. Therefore, it is critical that the assessment and 
curriculum be aligned so as to give the greatest benefit to fairness (Nielson, 1985). 

Problem Statement 

The issue at hand is what ^e of student achieves at a higher level? Do students with strong 
academic course work achieve at a higher level than those students that have a career and 
technical education (CTE) background? It seems that there is a concern with the progress of 
career and technical education students and how they perform on state mandated achievement 
tests versus students that are considered non-career and technical education. There are many 
other influences that affect student scores such as: ethnicity, SES, gender, parent’s education, 
rVEP, learning styles, etc. The purpose of this research project was to compare career and 
techmcal education students and non-career and technical education students on the Stanford9 
high stakes test. 



. Objectives 

The objective of the study was to compare the Stanford9 test scores between career and technical 
education and non-career and techmcal education students while controlling for extraneous 
variables, learning styles, special populations, gender, race, and ethnicity. 
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Definition of Terms 

understanding of this study, several terms commonly refeired to in education 

AJM^ The ^ronym for Arizona’s Instrument to Measure Standards. This high stakes test 
desi^^ '? be eligible for a high school graduation dipioma. 

Education: also known as vocational education. Organized educationai 
achvthes to offer a sequence of courses to provide individuals with the academic and 
techmcal knowledge and skills the individuals need to prepare for further education and for 
careers m a current or emerging employment sectors (other than careers requiring a 
baccalaureate, meter’s, or doctoral degree). The program includes competency-based 
applied learning that contributes to the academic knowledge, higher-order reasoning and 

problem-solymg skills, work attitudes, general employability skills, technical skills and 
occupational-specific skills. 

^ yeer ^d Technical Educational Prog rams: a coherent sequence of instruction designed to 
deliver the entire set of state-designated program competencies. A program is directly related 
to the preparation of individuals for employment in an occupation (e.g.,: mechanics, welding, 
federd^fhndl^^^ Approved and provisionally approved programs qualify for state and 

^amegieUnit: One unit of credit awarded in grades 9-12 for a minimum of 150 clock hours 
sch^^r "" of instruction during summer 



C ompetency: an educational “construct/concept” or abstraction derived from the workplace 
task, knowledge, skill, or attitude requirements. 

Competency Attainment: successful, demonstrated, and documented achievement of a 
competency. 

Conlmuous Improvement- The process of systematically planning and measuring program 
progress and results toward achieving desired improvements. 

C purse Sampler; A high school graduate who has completed a Level III career and technical 
education course but has not completed a career and technical education program per the 

distoct s planned sequence of courses. The course must be part of an approved career and 
tecnmcal education program. 



Individualized Vocational Education Plan (IVEP): required for each career and technical 
education student who is a member of a special populations category and who needs special 
services m order to succeed in his/her career and technical education program. The IVEP is a 
written plan that specifies the eligible category in which services are provided and lists the 
student s goals, sfiengths and weaknesses, and the services required to reach these goals. 
Examples of services, modifications or accommodations include: smaller classes, modification 

to classroom or equipment, alternative assessment instruments, additional time for completion 
or tutonng services. 



Local Education Agency /LEA): means a board of education or other legally constituted local 
school authonty having administrative control and direction of public elementaiy or secondaiy 
schools m a city, county, township, school district, or political subdivision in a State, or any 
other public educational institution or agency having administrative control and direction of a 
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career and technical education program. 

Non-standardized assessment: is the traditional form of assessment within the classroom. 
Placement, a measure of the number of career and technical education students who 
graduated and were placed in postsecondary education or advanced training, employment 
and/or military service within nine months. This measurement is required for program 
completers and optional for course samplers. 

Program CIP (Classification of I nstructional Programs) Code: the code used by Arizona to 
classify career and techmcal education programs. The code is derived from the U. S. 
Department of Education’s standard educational program classification system and includes a 
umform numbering system and description of educational programs. 

Program Concentrator: A student who completes two Carnegie units of a single VTE 
program. One Carnegie unit must be at level HI. 

Program Completer: A student who has attained at least 80% of the state designated 
competencies for an approved VTE program (levels I, II, HI). 

Special Populations: Eligible categories of students who may receive services in order to 
succeed in career and techmcal education courses and programs. Services provided must be 
documented on the student’s fVEP. Eligible categories include: 

Handicap/ Disabled: means an individual with any disability (as defined in section 3 of the 
Americans with Disabilities Act of 1990. (42 U.S.C. 12102)). 

LEP: a student with Limited English Proficiency: a secondary school student, an adult, or 
an out-of-school youth, who has limited ability in speaking, writing, or understanding the 
English language, and- 

A) Whom native language is a language other than English; or 

B) Who lives in a family or community environment in which a language other 
than English is the dominant language. 

Academically Disadvantaged : a student at or below the 25**' percentile on a standardized 
achievement or aptitude test whose secondary school grades are below 2 on a 4 scale, 
where 4.0 equals an A; and/or a student fails to attain minimal academic and career and 
technical education competencies. 

Economically Disadvantaged: a family or individual including foster children, which the 
Local Education Agency identifies as low income (on the basis of uniform methods 
described in the State plan) and who requires financial assistance to succeed in career and 
techmcal education. Examples of eligibility definitions include: Annual income at or below 
the official poverty line; Eligibility for Aid to Families with Dependent Children or other 
public assistance programs; and Eligibility for participation in programs assisted imder 
Title I of the WIA. 

Nontraditional: a student enrolled in a career and technical education program tha t has 
been identified as a nontraditional program for his or her gender. 

Single Parent: Individual who has custody and responsibility for the support and the care 
of one or more dependent children imder the age if eighteen in the same residence. The 
individual only providing child support, but does not provide custodial care, is not 
considered a single parent for eligibility purposes. Single pregnant women are included. 
Standardized assessment: is the form of assessment tha t is the same for all individuals in a 
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certain category (i.e., grade). 

Stanford9: The Ninth Edition of the Stanford Achievement Test used to fulfill the 
requirements of the Arizona Revised Statutes Section 5-741 through 744. It is a norm- 
referenced achievement test in the subjects of reading, language and mathematics. 
State:Designated Program Competencies: The career and technical education curriculum 
competencies identified as Levels I, II, and m for each career and technical education 

program available through the School To Work Division at the Arizona Department of 
Education. 

Unduplicated Enrollment: means that a student is counted only once even though he/she may 
meet more than one criterion. 



Conceptual/Theoretical Framework 

Multiple measures of assessment are typically used to yield valid results (Liu, 1997). At the 
intermediate level the standardized test is used to compare students with their peers fi-om similar 
leammg settings (Cooper, 1995). Agricultural Education is at a state in which there is question as 
to the purpose for an agricultural education program and it is aligned with career and technical 
education (CTE). Secondary CTE continues to suffer fi-om a negative image among students 
parents, educators, and policymakers (Wonacott, 2001). It is plagued by the negative stereotype 

that vocational education is only for non-college bound, the potential dropouts, and other students 
with vanous special needs. 

Admimstrators and Boards are in a position to determine the local status of an agricultural 
education program and the purpose of instruction in such course (Jewell, 1989). Career and 
technical education students and educators are constantly facing the challenge of new 
technologies, complex information, and skills. These changes place an overwhelming demand, 
that the students posses the ability to process vast amounts of information and knowledge in 
practical and systematic ways (Cano, 1993). The agricultural education programs, in Arizona, 
assist the students in developing and improving their problem-solving skills, which in turn aids in 
the process of critical thinking (Cano, 1993). These students may actually be at an enhanced level 
when talking about assessment. They are often skilled at taking various different forms of 
assessment as compared to the non-vocational student who may be exposed to one or two 
different forms. 

Regardless of the educational pathway, as a society, the custom is subscribe to the theory of “rite 
of passage (Sacks, 00). We learn, early in life, that the way to advance to the next grade or level 
is to pass the standardized test that is typically administered toward the end of the school year. 
Much like we have labeled students as “vocational” and “non-vocational,” test scores have been 
used as a lever for labeling a competent child as “overachievers” because his/her academic 
performance exceeded what the tests predicted. On the other hand, recall the hand wringing over 
the underachiever, the student whose brilliant test scores predicted greater things than what 
he/she actually accomplished (Sacks, 2000). These are often stigmas that stay with an individual 
(Sacks, 2000). Is this the way to teach? Should their test scores label students? 
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Standardized test are known to vary in their ability to assess the students knowledge fairly 
(Sanders & Horn, 1995). Vocational students may posses the hands-on skills for practical and 
systemic needs. Non-vocational students may also posses those same skills but are never 
instructed on the benefits and uses of those skills. Who is at the advantage and who is at the 
disadvantage? With the shift toward higher-order thinking one would think that the CTE student 
would be at the advantage over the non-vocational. Research indicates that students who study 
vocational agriculture in high school perform academically just as well as students who did not 
study vocational agriculture. It is fiirther researched that the ratios of vocational agriculture 
students who complete college is greater than that of students who did not have vocational 
agriculture in high school (Raven, 1990). 

Standardized tests are used to determine the students ability to learn. They are often referred to 
^ aptitude tests. Aptitude is defined to be a natural or acquired ability to understand new 
information ^d skills. It has been argued that these aptitude tests really measure the rate at 
which an individual is able to learn. The new theory for education of the twenty-first century is to 
concentrate on what is learned rather than what is taught (Brown, 1996). So should we use a 
standardized test to criticize and label or should we use them to praise what has been learned? 
These high stakes tests are often the most visible indicator of success and failure for the student as 
well as the parent (Nielson, 1985). Although they were developed from state standards that were 
agreed upon a committee, they fail to take into considerations that not every school and student is 
identical to the next. 



The state of Arizona has working curriculum standards (competencies) for the numerous CTE 
programs. These competencies enable the teacher/instructor to identify attainment of specific 
skills that are required by the students. This is a form of testing without the pressure of the high- 
stakes testing procedures. This type of instruction leads to instructionally relevant assessment 
versus irrelevant policy-driven evaluation. 

Tests have come to define our priorities. Schools are focusing on ’’core-subjects” and are forced 
to eliminate CTE and fine arts programs. These programs may not be directly correlated to the 
standards being assessed by the state mandated tests, but they do serve a purpose (see Figure 1). 

Students leam more efficiently when they are actively engage in a project or task. Career and 
Techmcal Education allows for various learning styles and exploration. A growing body of 
evidence indicates that learning styles are important to both understanding and managing the 
teaching-learning process (Cox & Zamudio, 1993). The fine arts discipline engage students in 
their own learning, allowing them to ask questions and explore. The most significant intellectual 
achievement is not so much in problem solving, but in question posing (Eisner, 2001). Students 
must be ^ided in their education not manipulated. The act of posing questions will in turn 
expand the ideas and understandings of the student. 
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Non-Vocational Students 



High School Graduation 



f 



Stanford9 Assessment 




Arizona Academic Standards 



Figure 1 Conceptual Framework for the Stanford9 Assessment. 
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Vocational Students 



There is too much labor in our schools and not enough work (Eisner, 2001). Work is defined as 
an effort for which you derive satisfaction (Eisner, 2001). Students should be motivated in school 
to the verge that they feel they are successfully accomplishing the task at hand. 

Our schools across the nation are becoming mass “test-prep” facilities. Schools under intense 
pressure to show better test results have allowed those tests to cannibalize the curriculum (Kohn, 
2001). Cumculum becomes narrowed as school district policies make it clear that what is to be 
tested is what is to be taught (Eisner, 2001). 

Methods/Procedures 

The static-group comparison, as described by Campbell and Stanley (1963), was used for this 
study. The static-group comparison design uses two groups of subjects (students). In this study 
the two groups compared are the CTE students and the non-CTE students. “This is a design in 
which a group which has experienced X is compared with one which has not, for the purpose of 
establishing the effect of X,” stated Campbell and Stanley (1963). The diagram of the static- 
group comparison is as follows: 

X O, 

O2 



Internal Validity 

The threats of history and testing were controlled because the students received only one test. 
They did not take part in a pretest/posttest study. The threats fi'om selection, mortality, and 
maturation were present because each individual student was not randomly selected and there was 
no pretest to determine if the dependent variable was affected by the independent variable. 

External Validity 

The experimental population of this study was high school students participating in the Stanford9 
test from three different geographical regions in the state of Arizona. The target population was 
all high school students engaged in completing the Stanford9 test for graduation requirements. 

Variables 

The dependent variable for this study was the scores received by the students on their Stanford9 
test. The independent variable of interest for this study was: 

1. 2000 graduates from an urban, suburban, and rural district in the state of Arizona. 

The independent variables with possible influence (extraneous) on the dependent variable were: 

1 . Gender: male/female 

2. Race / Ethnicity: White (not of Hispanic origin). Black (not of Hispanic origin), 

American Indian or Alaskan Native, Asian or Pacific Islander, Hispanic, Other / 
Multiracial. 

3. Special Populations (eligible and category): Handicapped, Limited English 

Proficiency, Economically Disadvantaged, Academically Disadvantaged, Single 
Parent. 
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4. Learning Styles. Visual Learners (Learn most comfortable by seeing), Auditory 

Learners (Le^ most comfortable by hearing), Kinesthetic Learners (Learn most 
comtortable by doing). 

5. Subject Selection: The schools that will be participating in the study were 

background. Several of the schools are form the rural areas of the 
state, while others represent the urban areas of Arizona. 

6. PopuMon and Sapling Procedure: Seven schools were chosen form across the state 

ot Anzona. The idea was to try to represent the state student population in a fair ’ 
and economic manner. 

Instrumentation 

^e S^ord9 instruments used in this study were developed for the Arizona Department of 
ucation. The second instrument used in this study was the “Learning Styles” assessment that 
was administered to all three districts participating in the study. Every sJent involved in Te 

s^dy was asked to coniplete the learmng styles assessment that was then calculated to arrive at 
the vanous learning styles of the targeted students. 

Data Collection Procedures 

The three school disrtcts selected for this study were asked to release their student test scores 
wth ^plete confidentiality. The test scores were then obtained through the Arizona 
Department of Education. 

Data Analysis 

rspcis m m were a^lyzed using the Statistical Package for the Social Sciences 

o7. ,• I / ■ ^ T analyzed using flequencies. means, and standard deviations, 
statistical tests used included correlations and regression. 

Results 

TOte students were the most prevalent group (see Table 1). Gender was divided almost equal 
between female and male (see Table 2). Students receiving IVEP services scored significantly 

Aem (see Table 3)^Table 4 indicates which variables were significantly associated with higher or 
lower test scores. The important vanable of interest, if a student is a CTE concentrator, was not 
significantly associated with either higher or lower test scores. 
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Table 1 
Race 





Frequency 


Percent 


White 


1547 


63.4 


Black 


65 


2.7 


Hispanic 


336 


13.8 


American Indian or 
Alaskan Native 


38 


1.6 


Asian or Pacific Islander 


63 


2.6 


Other Multiracial 


167 


6.8 


Missing 


224 


9.2 


Total 


2440 


100.0 



Table 2 
Gender 





frequency 


Percent 


male 


1215 


49.8 


female 


1223 


50.1 


Missing 


2 


.1 


Total 


2440 


100.0 



Table 3 

Comparing Ivep Groups on the Stanford9 Test Snnrps 



SERVICES 


'dumber 


Average Stanford9 test 
score* 


Standard 

Deviation 


Standard Erroi 
Mean 


services received 


60 


98.1500 


54.8034 


7.0751 


eligible but not received 


27 


64.2963 


40.2032 


7.7371 



* statistically significant at the 0.05 alpha level 
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Table 4 



Regression Analysis with Stanfnrd9 Test Scores the Denendent V.n'.hlp 



Variables 


oo UlC i^CDCIlUCIl 

Unstandardized Coefficients 


1 vanaoie 
Standardizec! 
Coefficients 


t-valu€ 


J Significance 




B 


Std. Erroi 


Beta 






(Constant or intercept] 


53.522 


14.248 




3.757 




Learning Styles Auditory Score 


1.298 


.218 


.107 


5.951 


.000* 


Learning Styles Visual Score 


5.396 


.279 


.364 


19.361 


.000* 


Learning Styles Kinesthetic Score 


-3.568 


.250 


-.268 


-14.262 


.000* 


Are they a Concentrator 


1.095 


2.721 


.008 


.403 


.687 


White female 


.148 


4.543 


.001 


.033 


.974 


White male 


-6.998 


4.591 


-.045 


-1.524 


.128 


Black female 


1 -24.501 


14.056 


-.032 


-1.743 


.081 


Black male 


-30.436 


10.434 


-.056 


-2.917 


.004* 


Hispanic female 


-22.636 


6.571 


-.075 


-3.445 


.001* 


Hispanic male 


-38.185 


6.061 


-.143 


-6.301 


.000* 


American Indian or Alaskan Native female 


-20.766 


15.412 


-.025 


-1.347 


.178 


American Indian or Alaskan Native male 


-27.220 


14.697 


-.034 


-1.852 


.064 


Asian or Pacific Islander female 


.722 


11.695 


.001 


.062 


.951 


Other female 


-6.982 


8.061 


-.017 


-.866 


.386 


Other male 

Mntp* A ciiin nr Donifir* TcIonrlck«> : i. 


-24.452 


7.891 


-.063 


-3.099 


.002* 



Note; Asian or Pacific Islander Males were included in the intercept (constant) and were' not significant. 



* statistically significant at the 0.05 alpha level 



Summary 

^1 five “Special Population” areas (handicapped, limited English proficiency, economically 
disadvantaged, academically disadvantaged and being a single parent) were significantly 
associated with lower test scores and were predominantly found in the CTE population. Higher 
Visum Le^er (learn by seeing) and Auditory Learner (learn by hearing) scores were significantly 
associated with higher test scores and were predominantly found with other students. Higher 
Kinesthetic Learner scores were significantly associated with lower test scores and were 
predominantly found with CTE students. Black, Hispanic or other males were associated with 
lower test scores. Hispanic females were associated with lower test scores. 

Grouping students according to their special population status affects the comparison because 
there was a higher proportion of CTE students who received special population services and 
because special population categories were associated with lower test scores. 

Implications/Recommendations 

Therefore, after controlling for the other influences (extraneous variables), no difference was 
found between the two groups. That means that they are just different groups of people and a 
raw score comparison is not an appropriate comparison. 

The conclusions are simple: Career and techmcal education students, for the most part, will 
dways do worse on raw score comparisons. When the appropriate extraneous variables are built 
into the equation and controlled for, there usually is no difference between CTE and other 
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students on standardized test scores. The raw score comparisons are inappropriate because the 
groups are different. The differences in scores can be attributed to the effects of the extraneous 
variables and not because of curriculum choice. 

The implications for educators are important. If extraneous effects are not understood and 
controlled for, than career and techmcal education will not have a very positive future. 

Therefore the recommendations are: CTE Administrators and Teachers must understand the 
problems associated with raw score comparisons on standardized tests. CTE state leaders must 
utilize this type of information in CTE promotional campaigns. 
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